Measuring impact of storyline engagement on health knowledge, attitudes, and norms: A digital evaluation of an online health-focused serial drama in West Africa

Background “Cest la Vie!” (CLV) is a serial drama that entertains, educates, and promotes positive health behaviors and social change for West African audiences. The purpose of this study was to evaluate if watching the CLV Season 2 series online had an impact on people’s health knowledge, attitudes, and norms, focusing on populations in francophone West Africa. Methods Between July 2019 and October 2019, viewers of CLV and non-viewers were recruited from Facebook and YouTube. We conducted an online longitudinal cohort study that assessed changes in health knowledge, attitudes, and norms (KAN) between these groups. Participants completed a baseline survey prior to the online airing and up to three follow-up surveys corresponding to specific health stories in the series, including sexual violence, emergency contraception, and female circumcision. We used descriptive statistics to describe viewers and non-viewers, and an item response theory (IRT) analysis to identify the effect of viewing CLV on overall KAN. Results A total of 1674 respondents participated in the study. One in four participants (23%, n = 388) had seen one of the three storylines from CLV Season 2 (ie, CLV viewers). At follow-up, viewers were more likely than non-viewers to know when to correctly use emergency contraception (P < 0.001) and to believe that the practice of female circumcision should end (P = 0.001). Compared to people who did not see CLV, viewers of the series had 26% greater odds of answering pro-health responses at follow-up about sexual assault, emergency contraception, and female circumcision. Further, the level of engagement with specific storylines was associated with a differential impact on overall outcome questions. Conclusions As internet access continues to grow across the globe and health education materials are created and adapted for new media environments, our study provides a novel approach to examining the impact of online entertainment-education content on health knowledge, attitudes, and norms.

Appendix S1. Graphical summaries used to prompt respondents for the three storylines evaluated: Sexual assault, emergency contraception, and female circumcision. After participants reviewed these graphical summaries, they were asked if they had seen this storyline. If they answered yes to seeing any of the storylines, they were treated as viewers; if they answered no to all storylines, they were treated as non-viewers.

Female circumcision
In this storyline from "C'est La Vie!": • Aïsha is sexually assaulted by a group of unknown men.
• Aïsha tell Rachel about the assault.
• Rachel accompanies Aïsha to the health center.
• At the health center, Aïsha is treated by Magar and Dr Moulaye.
In this storyline from "C'est La Vie!": • Rachel discusses sexual intercourse and consent with Magar.
• Rachel and Julien discuss consent before having sex.
• Rachel and Julien forgot to use a condom during sex.
• Rachel visits multiple pharmacies in search of the emergency contraception pill.
In this storyline from "C'est La Vie!": • There is a trial to determine who is responsible for the death of Magar's daughter, Caro, who died in Season 1 from complications related to female circumcision. • Rokoba, Touli, and la Féticheuse [FGM practitioner] are defendants. • The first trial takes place in the village of Jolal.
• The second trial takes place in the city of Ratanga. We model the CLV survey respondent data using an item response theory (IRT) approach. Specifically, for the pth respondent (p=1,...,1674) at the jth (j=0,1,2,3) visit we map the ith (i=1,...,21) survey item response to a {0,1} scale, creating the outcome variable Yipj . Yipj=1 indicates the respondent has ``correctly'' answered some question about KAN and Yipj=0 indicates they either did not know, or incorrectly answered the question. Our modeling objective will be to analyze the probability πipj = E[Yipj|-] conditional on covariates and latent terms we now introduce.
We include all subject level demographic information (see Table 1 variables) including a quadratic expansion of age at baseline, dummy variables for self-reported sex and education. We also included how many episodes of CLV Season 2 the respondent has seen and whether they like or do not like the show (variables not present in Table 1). This includes. We also include the time from baseline at which the subject has filled the survey in order to allow for comparisons with respondents across time and decompose the time-varying ``how much do you like CLV'' response into between and within subject components to estimate these effects separately.
Defining these variables in notation, let α be a global intercept and Zpj the vector containing all aforementioned subject-level covariates at measurement j and their corresponding regression coefficients δ. Let Xij (s+) be the {0,1} variable indicating subject i viewed any CLV storyline at follow-up j and β (s+) , it's corresponding effect. Similarly, for Model (2), let Xij (s,k) be the {0,1} variable indicating that subject i reports having engagement level k with story line s at measurement j and β (s,k) its corresponding effect. Denoting bi and bp as item and subject level latent intercepts, respectively, we complete our model specifications by linking covariates via the logistic function: Both bi and bp are modeled as parameters drawn from independent normal distributions with unknown variance: e.g. bi ~ N(0,s 2 item ). In the IRT literature, bi is referred to as the latent question difficulty and bp as the latent subject knowledge of whatever construct the items are designed to measure. In our context the latter would be an overall understanding of normative KAN after accounting for all subject covariates. In accordance with our goal of trying to understand the effect of public health media on KAN, the estimation of β (s+) in Model 1 and β (s,k) , (s = 1,2,3,k = 2,3,4) in Model 2 represent our primary goal of inference.
In Model 3, bsk represents the vector of question level effects specific to the sth storyline and the kth level of engagement, and bio is the intercept term representing the item difficulty for individuals who have not seen any CLV storyline.
In Model 4, we now estimate the probability of survey respondent, p ( p=1,…,388), at measurement j (j = 2,3), who has seen storyline s (s=1,2,3), responding that they would tell someone about the CLV series. We denote this probability as π'pjs and model this entity as a function of the same subject level covariates and effects discussed previously, Zpj and δ, respectively. Similar to before, we include a {0,1} indicator measurement for the subject having the kth engagement level with the storyline s, denoted Xpjs (k) .
The exponential of β (k) , the odds ratio, represents our inferential objective. We complete our model specification by including a subject-specific intercept bp, to adjust for within subject correlation across the follow-ups surveys: